microsoft open source blog
Introducing Distributed Data Parallel support on PyTorch Windows - Microsoft Open Source Blog
Model training has been and will be in the foreseeable future one of the most frustrating things machine learning developers face. It takes quite a long time and people can't really do anything about it. If you have the luxury (especially at this moment of time) of having multiple GPUs, you are likely to find Distributed Data Parallel (DDP) helpful in terms of model training. DDP performs model training across multiple GPUs, in a transparent fashion. You can have multiple GPUs on a single machine, or multiple machines separately.
Technology: Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.51)